Imaging apparatus and imaging method

ABSTRACT

An imaging apparatus and an imaging method that further facilitate recording of a video within an extraction range while moving the extraction range within an angle of view are provided. 
     An imaging apparatus according to an aspect of the present invention includes an image sensor that captures a reference video which is a motion picture, a housing that accommodates the image sensor, a detection portion for detecting a motion of the housing, and a processor. The processor is configured to execute setting processing of setting an extraction range smaller than an angle of view within the angle of view in a case of capturing the reference video, extraction processing of extracting an extraction video within the extraction range from the reference video, movement processing of moving the extraction range within the angle of view over time in accordance with the motion detected by the detection portion, and recording processing of recording the extraction video during movement of the extraction range in the movement processing on a recording medium.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of PCT International Application No.PCT/JP2020/028240 filed on Jul. 21, 2020, which claims priority under 35U.S.C. § 119(a) to Japanese Patent Application No. 2019-171661 filed onSep. 20, 2019. The above application is hereby expressly incorporated byreference, in its entirety, into the present application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an imaging apparatus that captures avideo which is a motion picture, and an imaging method using an imagingapparatus.

2. Description of the Related Art

Among imaging apparatuses capturing videos that are motion pictures,there is an imaging apparatus that extracts a designated range from animaging angle of view and records a video of the extraction range, suchas imaging apparatuses disclosed in JP2017-46355A and JP2019-22026A.

The imaging apparatuses disclosed in JP2017-46355A and JP2019-22026Ahave a function of changing a position or the like of the extractionrange within the angle of view at a predetermined speed. In a case wherethis function is used, for example, it is possible to track a subject inthe video or cause an angle of the recorded video to slide in a certaindirection without manually operating the imaging apparatuses duringimaging.

SUMMARY OF THE INVENTION

In a case of recording the video within the extraction range whilemoving the extraction range, it is required to reduce an effort ofperforming an input operation or the like by a user as much as possibleand more simply change the position or the like of the extraction range.

However, in the imaging apparatuses disclosed in JP2017-46355A andJP2019-22026A, an operation such as inputting various setting items isnecessary before a start of movement of the extraction range or duringmovement of the extraction range.

In view of the above matter, the present invention is conceived to solvethe following object.

An object of the present invention is to solve problems of thetechnology of the related art and provide an imaging apparatus and animaging method that further facilitate recording of a video within anextraction range while moving the extraction range within an angle ofview.

In order to accomplish the above object, an imaging apparatus accordingto an aspect of the present invention comprises an image sensor thatcaptures a reference video which is a motion picture, a housing thataccommodates the image sensor, a detector for detecting a motion of thehousing, and a processor configured to execute setting processing ofsetting an extraction range smaller than an angle of view within theangle of view in a case of capturing the reference video, extractionprocessing of extracting an extraction video within the extraction rangefrom the reference video, movement processing of moving the extractionrange within the angle of view over time in accordance with the motiondetected by the detector, and recording processing of recording theextraction video during movement of the extraction range in the movementprocessing on a recording medium.

According to the imaging apparatus according to the aspect of thepresent invention, the extraction range can be moved in accordance withthe motion of the housing detected by the detector. Thus, recording ofthe extraction video while moving the extraction range within the angleof view is further facilitated.

In addition, the imaging apparatus according to the aspect of thepresent invention may further comprise a display that displays theextraction video. In this case, it is more preferable that the processoris configured to, in the movement processing, move the extraction rangeand display the extraction video during movement of the extraction rangeon the display.

In addition, the processor may be configured to, in a case where thedetector has detected the motion, execute determination processing ofdetermining whether or not an end of the extraction range reaches an endof the angle of view, and the processor may be configured to execute themovement processing in a case where a determination that the end of theextraction range reaches the end of the angle of view is made in thedetermination processing.

Furthermore, it is more preferable that the processor is configured to,in a case where the detector has detected movement of the housing to afirst side of a horizontal direction as the motion, determine whether ornot an end of the extraction range positioned on a second side oppositeto the first side reaches an end of the angle of view positioned on thesecond side in the determination processing, and the processor isconfigured to move the extraction range to the first side in a casewhere a determination that the end of the extraction range reaches theend of the angle of view is made in the determination processing.

In addition, it is preferable that the processor is configured to, inthe movement processing, move the extraction range at a movement speedcorresponding to a time period from when the movement starts to when theend of the extraction range reaches the end of the angle of view.

In addition, the processor may be configured to, in a case where thedetector has detected the motion, select a subject captured within theextraction video before and after detection of the motion as a trackingtarget and, in the movement processing, move the extraction range suchthat the subject enters within the extraction range.

Furthermore, it is more preferable that the processor is configured to,in a case where the subject deviates from the angle of view duringexecution of the movement processing, finish the movement processing.

In addition, it is more preferable that the processor is configured to,in the movement processing, move the extraction range while changing asize of the extraction range based on a ratio of a size of a video ofthe subject to the size of the extraction range.

In addition, the processor may be configured to, in a case where thedetector has detected the motion of the housing during execution of themovement processing, further execute camera shake correction processingof performing camera shake correction on the extraction video duringmovement of the extraction range.

In addition, the processor may be configured to, in a case where thedetector has detected the motion of the housing in a direction oppositeto a movement direction of the extraction range during execution of themovement processing, finish the movement processing.

In addition, the processor may be configured to further execute zoomingprocessing of performing zooming on a video of a subject within theextraction video, and in a case where the zooming processing is executedduring execution of the movement processing, adjust a movement speed ofthe extraction range in accordance with the zooming in the movementprocessing.

In addition, according to an aspect of the present invention, an imagingapparatus comprising an image sensor that captures a reference videowhich is a motion picture, and a processor configured to execute settingprocessing of setting an extraction range smaller than an angle of viewwithin the angle of view in a case of capturing the reference video,extraction processing of extracting an extraction video within theextraction range from the reference video, zooming processing ofperforming zooming on a video of a subject within the extraction video,movement processing of moving the extraction range within the angle ofview over time at a movement speed corresponding to the zoomingperformed in the zooming processing, and recording processing ofrecording the extraction video during movement of the extraction rangein the movement processing on a recording medium can also beimplemented.

In addition, the imaging apparatus may further comprise a display thatdisplays the extraction video. In this case, it is more preferable thatthe processor is configured to, in the movement processing, move theextraction range and display the extraction video during movement of theextraction range on the display.

In addition, in the imaging apparatus, the zooming processing may beprocessing of performing electronic zooming on the video of the subjectby changing a size of the extraction range in accordance with a distancebetween the imaging apparatus and the subject within the extractionvideo, and the processor may be configured to, in the movementprocessing, move the extraction range at a movement speed correspondingto the electronic zooming.

In addition, in the imaging apparatus, the zooming processing may beprocessing of performing optical zooming on the video of the subjectwithin the extraction video by moving a lens of the imaging apparatusalong an optical axis, and the processor may be configured to, in themovement processing, move the extraction range at a movement speedcorresponding to the optical zooming.

In addition, according to an aspect of the present invention, an imagingmethod using an imaging apparatus including an image sensor thatcaptures a reference video which is a motion picture, the imaging methodcomprising a setting step of setting an extraction range smaller than anangle of view within the angle of view in a case of capturing thereference video, an extraction step of extracting an extraction videowithin the extraction range from the reference video, a movement step ofmoving the extraction range within the angle of view over time inaccordance with a motion of the imaging apparatus detected by adetector, and a recording step of recording the extraction video duringmovement of the extraction range in the movement step on a recordingmedium can also be implemented.

According to the imaging apparatus and the imaging method according tothe aspects of the present invention, recording of the video within theextraction range while moving the extraction range within the angle ofview is further facilitated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an exterior of an imaging apparatus according toone embodiment of the present invention in a view from a front surfaceside.

FIG. 2 is a diagram of the exterior of the imaging apparatus accordingto one embodiment of the present invention in a view from a rear surfaceside.

FIG. 3 is a diagram illustrating a configuration of an imaging apparatusmain body comprised in the imaging apparatus according to one embodimentof the present invention.

FIG. 4 is a descriptive diagram for a motion of the imaging apparatus.

FIG. 5 is a diagram illustrating a state of movement of an extractionrange in a case where a subject of a tracking target is present.

FIG. 6 is a diagram illustrating transitions of an extraction video inthe case illustrated in FIG. 5.

FIG. 7 is a diagram illustrating a state of movement of the extractionrange in a case where the subject of the tracking target is not present.

FIG. 8 is a diagram illustrating transitions of the extraction video inthe case illustrated in FIG. 7.

FIG. 9 is a diagram illustrating an example of a selection screen for amovement mode of the extraction range.

FIG. 10 is a descriptive diagram for determination processing.

FIG. 11 is a descriptive diagram for an angle-of-view end nearby area.

FIG. 12 is a diagram illustrating a processing flow in a case where atracking mode is selected (Part 1).

FIG. 13 is a diagram illustrating the processing flow in a case wherethe tracking mode is selected (Part 2).

FIG. 14 is a diagram illustrating a change in angle of view in a casewhere a user moves the imaging apparatus.

FIG. 15 is a diagram illustrating a processing flow in a case where apanning mode is selected (Part 1).

FIG. 16 is a diagram illustrating the processing flow in a case wherethe panning mode is selected (Part 2).

FIG. 17 is a diagram illustrating a configuration of an imagingapparatus main body according to another embodiment of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, a preferred embodiment (hereinafter, referred to as a firstembodiment) of the present invention will be described in detail withreference to the appended drawings. The embodiment described below ismerely an example for facilitating understanding of the presentinvention and does not limit the present invention. That is, the presentinvention may be changed or improved from the embodiment described belowwithout departing from a gist thereof. In addition, the presentinvention includes equivalents thereof.

In the present specification, a video is a motion picture, and themotion picture means a collection of a plurality of images (frameimages) consecutively captured at a constant frame rate.

<<Basic Configuration of Imaging Apparatus>>

A basic configuration of an imaging apparatus (hereinafter, referred toas an imaging apparatus 10) according to the first embodiment will bedescribed with reference to FIG. 1 to FIG. 4.

The imaging apparatus 10 is a portable imaging apparatus. Specifically,the imaging apparatus 10 is a digital camera having an exteriorillustrated in FIGS. 1 and 2 and is used for capturing a still pictureand a motion picture. In the present specification, a function ofcapturing a motion picture in real time will be mainly described amongfunctions of the imaging apparatus 10.

The imaging apparatus 10 is configured with an imaging apparatus mainbody 12 and a housing 14. The imaging apparatus main body 12 is a partof the imaging apparatus 10 excluding the housing 14. The housing 14 hasapproximately the same structure as a housing of a general digitalcamera and accommodates an imaging portion 20 and a processing portion30 illustrated in FIG. 3.

The imaging portion 20 captures a video and includes a lens unit 110, alens drive portion 120, a stop portion 130, a shutter 140, an imagesensor 150, and an analog/digital (A/D) converter 160 as illustrated inFIG. 3.

The lens unit 110 includes a zoom lens 112 and a focus lens 114. In thefirst embodiment, the zoom lens 112 is equipped with an anamorphic lens.Thus, in the first embodiment, the video can be captured within an angleof view (for example, an angle of view having a width-to-height ratio of2.35:1) that is wide in a lateral direction. The present invention isnot limited to the anamorphic lens, and imaging lenses such as a wideangle lens, a ultra wide angle lens, and a 360-degree lens may be used.In addition, the lens unit 110 may be configured to be interchangeablewith other lens units. In addition, the imaging portion 20 may comprisea plurality of lens units 110 having different angles of view.

Here, the angle of view in capturing the video using the image sensor150 is decided in accordance with specifications and the like of thelens unit 110 and the image sensor 150, and the video captured withinthe angle of view corresponds to a “reference video” according to theembodiment of the present invention.

The lens drive portion 120 is configured with a drive motor, notillustrated, and a drive mechanism, not illustrated, and moves thelenses of the lens unit 110 along an optical axis.

The stop portion 130 adjusts a size of an opening portion in accordancewith setting of a user or automatically, and adjusts a quantity of lightpassing through the opening portion.

The shutter 140 blocks light transmitted to the image sensor 150.

The image sensor 150 is configured with, for example, a charged coupleddevice (CCD) or a complementary metal oxide semiconductor image sensor(CMOS), forms an image by receiving light from a subject through thelens unit 110, and generates image data. Specifically, the image sensor150 converts a light signal received through a color filter into anelectric signal using a light-receiving element, amplifies the electricsignal using an auto gain controller (AGC), and generates analog imagedata from the signal after amplification.

The A/D converter 160 converts the analog image data generated by theimage sensor 150 into digital image data. The digital image datacorresponds to data of a frame image constituting the reference videothat is a motion picture.

The number of pieces of pixel data (that is, the number of pixels)constituting the digital image data is not particularly limited and isthe number of pixels greater than or equal to 10 million in the firstembodiment. Preferably, a lower limit of the number of pixels may begreater than or equal to 60 million. In addition, a preferred upperlimit of the number of pixels in the first embodiment is less than orequal to 1 billion and more preferably less than or equal to 5 billion.In a case where the number of pixels is above the lower limit,visibility of an extraction video (extraction video will be describedlater) extracted from the reference video can be secured. In addition,in a case where the number of pixels is below the upper limit, a pixelinformation amount of the reference video can be reduced, and a speed ofprocessing of the processing portion 30 is increased.

The processing portion 30 executes various processing related to theimaging apparatus 10 and includes a control processing portion 210 and avideo processing portion 220 in the first embodiment as illustrated inFIG. 3.

The processing portion 30 is configured with, for example, one or aplurality of processors and is specifically configured with a centralprocessing unit (CPU) and a control program. The present invention isnot limited thereto. The processor may be configured with a fieldprogrammable gate array (FPGA), a digital signal processor (DSP), anapplication specific integrated circuit (ASIC), a graphics processingunit (GPU), a micro-processing unit (MPU), or other integrated circuits(ICs), or may be configured with a combination thereof. In addition, asrepresented by a system on chip (SoC) or the like, the processor may besuch that functions of the entire system including the controlprocessing portion 210 and the video processing portion 220 areconfigured with one integrated circuit (IC) chip. Furthermore, ahardware configuration of each processor above may be implemented by anelectric circuit (circuitry) in which circuit elements such assemiconductor elements are combined.

The control processing portion 210 controls each portion of the imagingapparatus main body 12 in accordance with an operation of the user orautomatically, and, for example, can cause the imaging portion 20 tocapture (acquire) the reference video by controlling the imaging portion20. In addition, the control processing portion 210 can control thevideo processing portion 220 such that a video (specifically, theextraction video described later) is recorded on a recording medium. Inaddition, based on a contrast or the like of the entire or a part of animage indicated by the digital image data generated by the imagingportion 20, the control processing portion 210 can control the lensdrive portion 120 to set a focal point of the lens unit 110 to focus onthe subject in the image. In addition, based on brightness of the entireor a part of the image indicated by the digital image data generated bythe imaging portion 20, the control processing portion 210 can controlthe stop portion 130 and automatically adjust an exposure amount at atime of imaging.

The video processing portion 220 performs processing such as gammacorrection, white balance correction, and defect correction on thedigital image data generated by the imaging portion 20 and furthermore,compresses the processed digital image data in a compression formatcomplying with a predetermined standard. The video processing portion220 acquires the reference video from the compressed digital image datathat is sequentially generated during imaging, and executes variousprocessing on the acquired reference video.

In addition, the video processing portion 220 can set an extractionrange smaller than the angle of view within the angle of view of theimaging portion 20 and extract the extraction video (so-called croppedimage) captured within the extraction range from the reference video(for example, refer to FIG. 6 and FIG. 8). In addition, the videoprocessing portion 220 can record (picture recording) the extractionvideo on the recording medium under control of the control processingportion 210. The extraction video will be described in detail in a latersection.

In the following description, unless otherwise specified, processing ofeach of the control processing portion 210 and the video processingportion 220 will be described as processing performed by the processingportion 30 that is a processor according to the embodiment of thepresent invention. Processing performed by the processing portion 30will be described in detail in a later section.

The housing 14 further accommodates an internal memory 230 incorporatedin the imaging apparatus main body 12, a memory card 240 that isattachable to and detachable from the imaging apparatus main body 12through a card slot 260, and a buffer 250. The internal memory 230 andthe memory card 240 are the recording medium on which the extractionvideo is recorded, and are configured with a flash memory, aferroelectric memory, and the like. The recording medium may be in alocation other than the imaging apparatus main body 12. The processingportion 30 may record the video on an external recording medium throughwired or wireless communication.

The buffer 250 functions as a work memory of the processing portion 30and is configured with, for example, a dynamic random access memory(DRAM) or a ferroelectric memory.

A display 40 is attached to a rear surface of the housing 14 asillustrated in FIG. 2. The display 40 is configured with a liquidcrystal display (LCD), an organic electroluminescence (EL) display, alight emitting diode (LED) display, an electronic paper, or the like.The display 40 displays the extraction video and also appropriatelydisplays a setting screen for an imaging condition and the like, ascreen for mode selection, and the like.

The housing 14 comprises an operating part 50. The user performs variousoperations related to imaging through the operating part 50. Forexample, as illustrated in FIGS. 1 and 2, the operating part 50 includesa release button 310, a zoom lever 320, operation dials 330 and 340, anda cross button 350 arranged on an outer surface of the housing 14. In acase where these devices are operated by the user, the devices transmitvarious control signals toward the processing portion 30. For example,in a case where the zoom lever 320 is operated, a control signal forchanging a zoom magnification is transmitted toward the processingportion 30 in accordance with the operation. The processing portion 30controls the lens drive portion 120 to be driven in order to move thezoom lens 112 in accordance with the control signal.

In addition, in the first embodiment, the display 40 is a touch paneldisplay and doubles as the operating part 50. In a case where the usertouches a screen of the display 40, a control signal corresponding tothe touched position is transmitted toward the processing portion 30.For example, in a case where a mode selection screen (refer to FIG. 9)described later is displayed on the display 40, and the user touches onecorresponding button of a plurality of mode selection buttons drawn onthe screen, a control signal for setting a selected mode is transmittedtoward the processing portion 30. In addition, the user can instruct theprocessing portion 30 to execute zooming processing or shutterprocessing (imaging processing) through an operation of touching apredetermined location on the screen of the display 40.

In the first embodiment, a detector 60 is attached to the housing 14.The detector 60 detects a motion of the imaging apparatus 10 includingthe housing 14 and specifically, detects movement of the housing 14 in ahorizontal direction. As illustrated in FIG. 4, movement of the housing14 in the horizontal direction is a motion of the housing 14 in a casewhere the user moves (in a strict sense, rotationally moves) the imagingapparatus 10 about a vertical axis C as a center such that a directionof a front surface of the imaging apparatus 10 changes in order tochange the angle of view of imaging. FIG. 4 is a diagram of the imagingapparatus 10 viewed from above.

For example, the detector 60 according to the first embodiment isconfigured with a gyro sensor attached to the housing 14, measures anangular velocity, a rotation amount (in FIG. 4, denoted by referencenumeral 0), and a movement direction in a case where the housing 14moves in the horizontal direction, and detects the motion of the housing14 based on the measurement result. The detector 60 is not limited toonly the gyro sensor and may be configured with a motion sensor otherthan the gyro sensor, for example, an acceleration sensor. In addition,the motion of the housing 14 may be detected by determining whether ornot the angle of view is changed by analyzing the reference video usingan image processing apparatus as the detector 60.

<<Extraction Video>>

Next, the extraction video will be described in detail with reference toFIGS. 5 to 8.

States illustrated in each of FIGS. 5, 6, and 8 transition in order of astate in an upper part, a state in a middle part, and a state in a lowerpart. In addition, in the drawings from FIG. 5, the angle of view at atime of capturing the reference video is designated by reference numeralA1. The extraction range is designated by reference numeral A2. Thereference video is designated by reference numeral P1. The extractionvideo is designated by reference numeral P2.

The extraction video is a video set to be smaller than the angle of viewand is extracted from the reference video and displayed on the display40. In the first embodiment, the reference video is a high image qualityvideo consisting of 10 million or more (preferably, 60 million or more)pixels. Thus, the extraction video extracted from the reference video isalso a video having sufficiently high image quality. In the firstembodiment, as illustrated in FIGS. 6 and 8, outer edges of theextraction video have an oblong shape. However, the shape of the outeredges is not particularly limited and may be a square shape, aparallelogramic shape, a trapezoidal shape, a rhombic shape, a circularshape or an elliptical shape, a triangular shape or a polygonal shape ofa pentagonal or higher-faceted shape, or an indefinite shape.

A position, a size, a width-to-height ratio (aspect ratio), and the likeof the extraction range are initially set. Normally, a predeterminedrange within the angle of view is set as the extraction range. Theposition, the size, the width-to-height ratio, and the like of theextraction range may be set and changed on the user side. For example, asetting screen not illustrated may be displayed on the display 40, andthe user may set and change the position, the size, the width-to-heightratio, and the like of the extraction range through the setting screen.

In the first embodiment, in a case where a predetermined condition issatisfied during imaging, the processing portion 30 moves the extractionrange within the angle of view over time. Here, “moving over time” meansmoving the extraction range relative to the angle of view such that theposition of the extraction range gradually changes, and may include acase where movement is stopped (paused) in the middle of imaging.

By moving the extraction range over time during imaging, for example,the subject captured within the extraction range can be set as atracking target, and the subject can be tracked within the angle of viewas illustrated in FIG. 5. That is, by using a function of the processingportion 30, the extraction range can be automatically moved inconnection with movement of the subject within the angle of view suchthat the subject falls within the extraction range.

In addition, in a case where the subject of the tracking target moves ina depth direction (that is, a direction of approaching to and separatingfrom the imaging apparatus 10), a size of a video of the subject changesas illustrated in FIG. 5. That is, a ratio (hereinafter, referred to asa subject video ratio) of the size of the video of the subject to thesize of the extraction range changes. Thus, in the first embodiment, asillustrated in FIG. 5, the size of the extraction range is changed inaccordance with movement of the subject in the depth direction, that is,a change in subject video ratio. Specifically, the size of theextraction range is adjusted such that the subject video ratio isapproximately constantly maintained. This size adjustment corresponds toelectronic zooming processing described later.

According to the above description, as long as the subject of thetracking target is present within the angle of view, the extractionvideo including the subject of the tracking target can be displayed onthe display 40 at all times as illustrated in FIG. 6. In addition, inthe displayed extraction video, the size of the subject of the trackingtarget is approximately constantly maintained. In addition, the userdoes not need to move the imaging apparatus 10 in order to track thesubject of the tracking target. Thus, the angle of view is not manuallychanged, and distortion (blurred video or the like) of the videooccurring at a time of manually changing the angle of view can beavoided. Such an effect is particularly effective in a case where theangle of view is relatively increased by using an anamorphic lens or thelike.

Meanwhile, in a case where the subject captured within the extractionrange is not the tracking target (for example, in a case where a stillobject such as a landscape is the subject), the extraction rangeautomatically slides (pans) from one end to the other end of the angleof view in the lateral direction as illustrated in FIG. 7. Accordingly,a video of which an imaging angle is continuously changed in thehorizontal direction as illustrated in FIG. 8, that is, a panoramicmotion picture, can be displayed on the display 40. In addition, theuser does not need to move the imaging apparatus 10 in order to changethe imaging angle. Thus, the angle of view is not manually changed, anddistortion (blurred video or the like) of the video occurring at a timeof manually changing the angle of view can be avoided. Such an effect isparticularly effective in a case where the angle of view is relativelyincreased by using an anamorphic lens or the like.

A movement speed of the extraction range may be set in advance or may beable to be changed to any speed by causing the user to perform, forexample, a button operation during movement of the extraction range. Inaddition, a time period from when movement of the extraction rangestarts to when an end of the extraction range reaches an end of theangle of view, that is, a required time period for movement, may beobtained. In this case, a remaining movement time period tr based on adifference between the obtained required time period and an elapsed timeperiod from the start of movement may be displayed on the display 40 asillustrated in FIG. 8.

In addition, automatic movement (panning) of the extraction range basedon the function of the processing portion 30 and a movement operation(operation for changing the angle of view) of the imaging apparatus 10performed by the user may be performed at the same time.

As described above, a movement mode of the extraction range includes afirst mode (hereinafter, referred to as a tracking mode) in which theextraction range is moved to track the subject of the tracking target,and a second mode (hereinafter, referred to as a panning mode) in whichthe extraction range is moved in a certain direction. For example, theuser can select any one mode on the mode selection screen illustrated inFIG. 9, and the processing portion 30 moves the extraction range withinthe angle of view in accordance with the mode selected by the user.

The movement mode of the extraction range may be designated based on anintention of the user, specifically, a selection result on the modeselection screen illustrated in FIG. 9.

Alternatively, the imaging apparatus 10 may automatically determinewhether or not the tracking target is present by analyzing the referencevideo, and automatically designate the movement mode based on thedetermination result. In this case, for example, the tracking mode isselected in a case where the subject of the tracking target is present,and the panning mode is selected in a case where the subject of thetracking target is not present.

Alternatively, in a case where the user moves the imaging apparatus 10,the detector 60 may detect a movement speed (specifically, an angularvelocity) or the like and automatically designate the movement modebased on the detection result. In this case, for example, the trackingmode is selected in a case where the movement speed is random, and thepanning mode is selected in a case where the movement speed is constant.

<<Processing Performed by Processing Portion>>

Next, processing of the processing portion 30 will be described.Processing of the processing portion 30 is broadly divided intoprocessing executed by the control processing portion 210 and processingexecuted by the video processing portion 220. For example, examples ofthe former processing include imaging processing, autofocus processing,optical zooming processing, and optical camera shake correction.Examples of the latter processing include electronic zooming processing,electronic camera shake correction processing, setting processing,extraction processing, determination processing, movement processing,recording processing, and alert processing. Hereinafter, a summary ofeach processing will be described.

[Imaging Processing]

The imaging processing is processing in which the processing portion 30captures a video within the angle of view of the imaging portion 20,that is, the reference video, by controlling the imaging portion 20. Forexample, in a case where the imaging apparatus 10 is started, theimaging processing is automatically started along with the start of theimaging apparatus 10.

[Autofocus Processing]

The autofocus processing is processing in which the processing portion30 controls the lens drive portion 120 such that the focus lens 114 ofthe lens unit 110 moves along the optical axis. For example, theautofocus processing is executed in a timely manner during movement ofthe extraction range. Accordingly, the subject within the movingextraction range can be focused.

[Optical Zooming Processing]

The optical zooming processing is an example of the zooming processingand is processing of performing optical zooming on the video of thesubject within the extraction video. Specifically, in a case where theuser operates the zoom lever 320 as a zoom operation part, the opticalzooming processing is executed, and the processing portion 30 moves thezoom lens 112 along the optical axis by controlling the lens driveportion 120. Other examples of the zoom operation part include arotation ring for a zoom operation in the lens unit 110. The opticalzooming processing is used together with the electronic zoomingprocessing. For example, in a case where image quality of the extractionvideo deteriorates (becomes coarse) by frequently executing theelectronic zooming processing during imaging, the optical zoomingprocessing is appropriately executed in order to improve the imagequality.

[Optical Camera Shake Correction]

The optical camera shake correction processing is an example of camerashake correction processing and is processing for stabilizing theextraction video by suppressing an effect of vibration or the likeapplied to the imaging apparatus 10 during imaging. Specifically, theprocessing portion 30 shifts the optical axis by controlling the lensdrive portion 120 to move the lens unit 110. The optical camera shakecorrection is implemented using a well-known technology. For example, atechnology for optical camera shake correction disclosed in JP5521518Bcan be used.

[Electronic Zooming Processing]

The electronic zooming processing is an example of the zoomingprocessing and is processing of performing electronic zooming (digitalzooming) on the video of the subject within the extraction video.Specifically, the processing portion 30 changes the size of theextraction range (in other words, the number of pixels of the extractionvideo) within the angle of view. For example, the electronic zoomingprocessing is executed in a timely manner during movement of theextraction range. For example, in a case where the subject within theextraction range moves in the depth direction of the imaging apparatus10, and the size of the video of the subject of the tracking targetchanges, the electronic zooming processing is executed in order toadjust the subject video ratio.

In the electronic zooming processing, the size of the extraction rangeis changed in accordance with a distance between the imaging apparatus10 and the subject within the extraction video. The distance between theimaging apparatus 10 and the subject within the extraction video can bemeasured using a distance measurement technology with which a generaldigital camera is equipped. For example, a distance measurement methodbased on a phase difference method using a phase difference pixel, or amethod of measuring a distance to the subject based on a flight timeperiod of light using a time of flight (TOF) method can be used.

In addition, according to the electronic zooming processing, the size(display size) of the subject of the tracking target can be constantlymaintained during movement of the extraction range by analyzing theextraction video.

[Electronic Camera Shake Correction Processing]

The electronic camera shake correction processing is an example of thecamera shake correction processing and is processing in which theprocessing portion 30 shifts the position of the extraction range inorder to stabilize the extraction video by suppressing the effect ofvibration or the like applied to the imaging apparatus 10 duringimaging. Electronic camera shake correction is implemented using awell-known technology, for example, a technology for electronic camerashake correction (electric image stabilization; EIS) disclosed inJP5521518B, WO2016/181717A, or the like. In executing the electroniccamera shake correction processing, an immediately previous frame imageamong frame images constituting the reference video is stored in thebuffer 250 during imaging. In a stage in which a new frame image iscaptured, the processing portion 30 obtains an amount of shift in imagewithin the extraction range by comparing the frame image with theimmediately previous frame image stored in the buffer 250. In a casewhere the obtained amount of shift exceeds a threshold value, theelectronic camera shake correction processing is executed.

[Setting Processing]

The setting processing is processing in which the processing portion 30sets the extraction range within the angle of view in the referencevideo. Normally, the setting processing is executed in a case where theimaging apparatus 10 is started. At this point, in the settingprocessing, the position, the size, the width-to-height ratio (aspectratio), and the like of the extraction range are set to initial values.However, the present invention is not limited thereto. The user may setthe position, the size, the width-to-height ratio, and the like of theextraction range through the operating part 50 and appropriately changethe position, the size, the width-to-height ratio, and the like of theextraction range after setting.

In addition, in a case where the tracking mode is selected, theextraction range is set based on the subject of the tracking target. Forexample, the user moves the imaging apparatus 10 such that one subjectpresent within the angle of view is displayed on the display 40 for acertain time period or longer. Specifically, the user changes the angleof view or performs zooming. In this case, the processing portion 30recognizes the subject as the subject of the tracking target and setsthe extraction range into which the subject falls. A procedure otherthan the above procedure may be used as a procedure of setting theextraction range based on the subject. For example, the entire screen ofthe reference video may be displayed on the display 40, and the user mayset the extraction range into which the subject falls, by touching thesubject captured in the reference video.

[Extraction Processing]

The extraction processing is processing in which the processing portion30 extracts (that is, cutting out) the extraction video captured withinthe extraction range from the reference video. For example, in a casewhere the tracking mode is selected, a video of a certain rangeincluding the video of the subject of the tracking target is extractedas the extraction video. In addition, in a case where the panning modeis selected, the extraction video is extracted such that the subjectsequentially changes (slides). The extraction video is displayed on thedisplay 40 as illustrated in FIGS. 6 and 8, and the user can check theextraction video through the display 40.

[Determination Processing]

The determination processing is processing in which the processingportion 30 determines whether or not the end of the extraction rangereaches the end of the angle of view by the motion of the housing 14 ina case where the detector 60 has detected the motion. In the firstembodiment, the determination processing is executed in a case where thedetector 60 has detected movement of the housing 14 in the horizontaldirection as the motion of the housing 14 in a state where the panningmode is selected.

The determination processing will be specifically described withreference to FIG. 10. In the determination processing, in a case wherethe user moves the imaging apparatus 10 in the horizontal direction (inFIG. 10, a direction indicated by a bold arrow), the detector 60 detectsthe movement of the housing 14 at the time of movement. At this point, aside to which the imaging apparatus 10 heads at a time of movement inthe horizontal direction will be referred to as a first side, and theopposite side will be referred to as a second side. In the caseillustrated in FIG. 10, a left side corresponds to the first side, and aright side corresponds to the second side.

In a case where the detector 60 has detected movement of the housing 14to the first side of the horizontal direction, the processing portion 30shifts the extraction range to the second side of the horizontaldirection by an amount corresponding to a movement amount of the housing14 using the electronic camera shake correction processing (EIS).Accordingly, as illustrated in FIG. 10, the extraction range moves tothe second side relative to the angle of view. The processing portion 30executes the determination processing and determines whether or not theend of the extraction range positioned on the second side reaches theend of the angle of view positioned on the second side by the relativemovement. A determination result of the determination processing isreflected on whether or not to execute the movement processing.

[Movement Processing]

The movement processing is executed during imaging and is processing inwhich the processing portion 30 moves the extraction range within theangle of view over time. In the first embodiment, in a case where themotion of the housing 14 is detected by the detector 60, the movementprocessing is executed in accordance with the detected motion. Arelationship between the motion of the housing 14 detected by thedetector 60 and execution of the movement processing will be describedin detail in a later section.

In addition, during execution of the movement processing, the processingportion 30 displays the extraction video during movement of theextraction range on the display 40. During execution of the movementprocessing, in a case where the housing 14 moves (vibrates) due to, forexample, transmission of vibration to the imaging apparatus 10 from theuser or a ground, the electronic camera shake correction processing orthe optical camera shake correction processing is executed on theextraction video during movement of the extraction range. Accordingly,blurriness of the extraction video displayed on the display 40 isreduced, and the displayed video is stabilized.

A method of moving the extraction range in the movement processingvaries depending on the selected mode out of the tracking mode and thepalming mode. In a case where the tracking mode is selected, theprocessing portion 30 sets the subject captured within the extractionrange as the tracking target. In the movement processing, the processingportion 30 moves the extraction range such that the subject of thetracking target enters within the extraction range (in a strict sense,falls within the extraction range). In such a manner, in the movementprocessing in a case where the tracking mode is selected, the subject ofthe tracking target is searched for within the angle of view, and theextraction range is moved in connection with movement of the subject ofthe tracking target.

An algorithm for searching for the subject of the tracking target withinthe angle of view is not particularly limited. As an example of thealgorithm, an image of the subject set as the tracking target may bestored in the buffer 250 as a template image, and a video of a partmatching with the template image may be specified by comparing thetemplate image with the reference video by applying a well-knowntemplate matching technology.

In addition, in the movement processing in a case where the trackingmode is selected, it is preferable that the movement speed of theextraction range is set to a speed corresponding to a movement speed ofthe subject so that the subject of the tracking target can be tracked.In a case where the angle of view is divided in units of pixels, themovement speed of the extraction range is the number of pixels throughwhich the moving extraction range passes within a unit time period.

Meanwhile, in the movement processing in a case where the panning modeis selected, the extraction range is moved from one end to the other endof the angle of view in the lateral direction. At this point, themovement speed of the extraction range may be a constant speed or, forexample, may be a speed corresponding to a movement speed (specifically,an angular velocity) of movement of the housing 14 in the horizontaldirection detected by the detector 60. In the movement processing in acase where the panning mode is selected, the user may be able to changethe movement speed of the extraction range by, for example, operatingthe operating part 50 during execution of the movement processing (thatis, during movement of the extraction range).

As additional description to the movement speed of the extraction range,in a case where the electronic zooming processing or the optical zoomingprocessing is executed during execution of the movement processing, thesize of the extraction range (that is, the number of pixels of theextraction video) changes. Thus, in a case where the movement speed ofthe extraction range is the same before and after the zoomingprocessing, the movement speed of the extraction range looks rapidlychanging for the user viewing the extraction video through the display40. Therefore, in the first embodiment, in a case where any of thezooming processing is executed during execution of the movementprocessing, the processing portion 30 adjusts the movement speed of theextraction range in accordance with the zooming (specifically, the zoommagnification after change) in the movement processing. Specifically,the processing portion 30 decreases the movement speed in a case ofzooming up and increases the movement speed in a case of zooming out.Accordingly, an awkward look of the extraction video can be solved.

Returning to description of the movement processing, the processingportion 30 finishes the movement processing in a case where any offinish conditions (J1) to (J5) below is established.

(J1) In a case where the user issues an instruction to finish themovement processing through the operating part 50 during execution ofthe movement processing.

(J2) In a case where the user moves the housing 14 in a directionopposite to a movement direction of the extraction range duringexecution of the movement processing, and the detector 60 detects themotion.

(J3) In a case where the tracking mode is selected, and the subject ofthe tracking target deviates from the angle of view during execution ofthe movement processing.

(J4) In a case where the tracking mode is selected, and the subject ofthe tracking target stops (stands still) for a certain time period orlonger during execution of the movement processing.

(J5) In a case where the panning mode is selected, and the end of theextraction range reaches the end of the angle of view during executionof the movement processing.

A finish condition of the movement processing is not limited to the fiveconditions. Conditions other than the five conditions may be furtherincluded, and one to four of the five conditions may not be included.

[Recording Processing]

The recording processing is processing in which the processing portion30 records the extraction video on the recording medium including theinternal memory 230, the memory card 240, and the like. For example, therecording processing is started at a point in time when the releasebutton 310 is pushed once, and is finished at a point in time when therelease button 310 is pushed again. In the first embodiment, in a casewhere the movement processing is executed, the processing portion 30records the extraction video during movement of the extraction range inthe movement processing on the recording medium in the recordingprocessing. In addition, in a case where at least one of the opticalzooming processing, the electronic zooming processing, the opticalcamera shake correction processing, or the electronic camera shakecorrection processing is executed during execution of the movementprocessing, the extraction video after the processing is performed isrecorded in the recording processing. In the first embodiment, in a casewhere the movement processing is finished, the recording processing isfinished in connection with the finish of the movement processing.

In the first embodiment, only the extraction video is recorded on therecording medium in the recording processing, and other videos(specifically, the reference video) are not recorded and are discarded.Accordingly, a used capacity of the recording medium can be reduced byan amount obtained by not recording the reference video. However, thepresent invention is not limited thereto. Both of the extraction videoand the reference video may be recorded on the recording medium.

[Alert Processing]

The alert processing is processing in which the processing portion 30issues an alert toward the user in a case where a predeterminedsituation is reached during execution of the movement processing. Forexample, in executing the movement processing in the tracking mode, in acase where the moving extraction range approaches an angle-of-view endnearby area within the angle of view, the alert processing is performedin order to notify the user of the approach. As illustrated in FIG. 11,the angle-of-view end nearby area corresponds to a range from a positionseparated from the end of the angle of view by a predetermined distanceto the end in the angle of view, and is a range with hatching in FIG.11.

An alert method in the alert processing is not particularly limited. Forexample, examples of the alert method include displaying an alertmessage or changing an outer edge frame of the extraction video to apredetermined color (specifically, red or the like).

<<Flow of Movement Processing>>

In the first embodiment, the video is captured using an imaging methodaccording to the embodiment of the present invention. In a case wherethe detector 60 has detected the motion of the housing 14 duringimaging, the processing portion 30 executes the movement processing inaccordance with the detected motion of the housing 14. Specifically, inthe first embodiment, in a case where the user moves the imagingapparatus 10 for the purpose of, for example, changing the angle ofview, the detector 60 detects the motion of the housing 14 at the timeof movement, and the processing portion 30 executes the movementprocessing based on the detection as a trigger.

In the movement processing, the extraction range moves to track thesubject or automatically toward the end of the angle of view.Accordingly, after the detector 60 detects the motion of the housing 14,the user does not need to move the imaging apparatus 10 again in orderto change the angle of view. Consequently, the user can concentrate onimaging. That is, recording of the extraction video while moving theextraction range is further facilitated for the user.

A flow of various processing including the movement processing variesdepending on the selected mode out of the tracking mode and the panningmode. Hereinafter, the flow of processing will be described for eachmode.

(Flow of Processing in Case where Tracking Mode is Selected)

The flow of processing in a case where the tracking mode is selectedwill be described with reference to FIGS. 12 to 14.

In executing the movement processing, the user first starts the imagingapparatus 10. After the start of the apparatus, the processing portion30 executes the setting processing (S001) and sets the extraction rangewithin the angle of view in accordance with the initial values or inputvalues and the like of the user. Step S001 corresponds to a settingstep.

Then, capturing of the reference video is started within the angle ofview of the imaging apparatus 10, and the processing portion 30 extractsthe extraction video from the reference video by executing theextraction processing (S002). Step S002 corresponds to an extractionstep. In addition, the extraction video is displayed on the display 40by executing the extraction step (S003).

The user pushes the release button 310 in order to start recording theextraction video. Accordingly, the processing portion 30 executes therecording processing and starts recording the extraction video (S004).In other words, a recording step is started by pushing the releasebutton 310.

Furthermore, in a case where the tracking mode is selected, the usermoves the imaging apparatus 10 in order to track a certain subject. Forexample, in a case where a subject J illustrated in FIG. 14 moves, theuser moves the imaging apparatus 10 such that the subject J continuouslyenters within the extraction range as illustrated in the drawing. Thedetector 60 detects the motion of the housing 14 at the time of movement(S005). An upper part of FIG. 14 illustrates the angle of viewimmediately before the imaging apparatus 10 is moved, and a lower partof FIG. 14 illustrates the angle of view immediately after the imagingapparatus 10 is moved.

In a case where the detector 60 detects the motion of the housing 14,the processing portion 30 compares the extraction videos before andafter detection of the motion by the detector 60 and selects the subjectJ captured within both of the extraction videos in common as thetracking target (S006). At this point, the processing portion 30 canrecognize the subject J captured in the extraction videos before andafter detection of the motion by the detector 60, using the templatematching technology or the like. A method of selecting the trackingtarget is not limited to the above method. The user may designate thesubject by, for example, touching the video of the subject on thedisplay 40, and the tracking target may be selected based on thedesignation operation. Alternatively, the extraction videos before andafter detection of the motion by the detector 60 may be compared, and asubject having the smallest movement amount among subjects captured ineach extraction video may be selected as the tracking target.

Then, the processing portion 30 executes the movement processing andmoves the extraction range within the angle of view such that thesubject J enters within the extraction range as illustrated in FIG. 5(S007). Step S007 corresponds to a movement step. In the movement step,the processing portion 30 may move the extraction range at a movementspeed set in advance or may move the extraction range at a movementspeed (for example, the same speed as the subject J) corresponding to aspeed of the subject J calculated by analyzing the extraction video.

In the movement processing, the processing portion 30 moves theextraction range and displays the extraction video during movement ofthe extraction range on the display 40 (refer to FIG. 6). In addition,in the recording processing executed at the same time as the movementprocessing, the processing portion 30 records the extraction videoduring movement of the extraction range in the movement processing onthe recording medium. That is, the recording step of recording theextraction video during movement of the extraction range in the movementstep is executed.

In addition, in a case where the detector 60 detects the motion(specifically, a motion caused by transmission of vibration or the like)of the housing 14 during execution of the movement processing (S008),the processing portion 30 executes the electronic camera shakecorrection processing or the optical camera shake correction processing(S009). In step S009, camera shake correction is performed on theextraction video during movement of the extraction range. Accordingly,the extraction video that changes along with movement of the extractionrange can be stabilized, and the extraction video can be smoothlychanged even in a case where a camera shake or the like occurs.

In addition, in a case where a ratio (that is, the subject video ratio)of a size of a video of the subject J to the size of the extractionrange changes due to movement of the subject J in the depth direction,the amount of change may exceed a threshold value (S010). In this case,the processing portion 30 executes the electronic zooming processing andchanges the size of the extraction range in accordance with a distancebetween the imaging apparatus 10 and the subject J (S011). In such amanner, the processing portion 30 moves the extraction range whilechanging the size of the extraction range based on a change in subjectvideo ratio in the movement processing. Accordingly, for example, it ispossible to move the extraction range while maintaining the subjectvideo ratio. Consequently, while the subject J is tracked, the video ofthe subject J can be displayed on the display 40 in an approximatelyconstant size as illustrated in FIG. 6.

In a case where execution of the electronic zooming processing(particularly, execution of the zooming up) causes the size of theextraction range to be less than a predetermined size or causes thenumber of pixels included in the extraction range to be less than apredetermined number (S012), the image quality of the extraction videobecomes coarse. Thus, the processing portion 30 executes the opticalzooming processing and performs the optical zooming on the video of thesubject within the extraction video (S013). Accordingly, the imagequality of the extraction video decreased by execution of the electroniczooming processing can be recovered. In step S013, a notification thatprompts the user to perform the optical zooming processing may bedisplayed on the display 40.

In addition, in a case where the electronic zooming processing or theoptical zooming processing is executed during execution of the movementprocessing, the processing portion 30 adjusts the movement speed of theextraction range in accordance with the electronic zooming or theoptical zooming (S014). In the movement processing after S014, theprocessing portion 30 moves the extraction range at the adjustedmovement speed, that is, a movement speed corresponding to theelectronic zooming or the optical zooming. Accordingly, inconvenience(specifically, a situation in which the movement speed of the extractionrange looks rapidly changing for the user viewing the extraction videothrough the display 40) accompanied by execution of the zoomingprocessing can be avoided.

In addition, in the movement processing, the processing portion 30determines whether or not the moving extraction range approaches theangle-of-view end nearby area within the angle of view (S015). In a casewhere the extraction range approaches the angle-of-view end nearby area,the processing portion 30 executes the alert processing (S016).Accordingly, the user can be prompted to perform an operation forchanging the angle of view, an operation for issuing an instruction tofinish imaging, or the like by notifying the user that the extractionrange approaches near the end of the angle of view.

In the processing described so far, steps S007 to S016 are repeatedlyexecuted during the movement processing. The processing portion 30finishes the movement processing at a point in time when any of thefinish conditions (J1) to (J4) is established during execution of themovement processing (S017). At this point, the recording processing maybe finished in accordance with the finish of the movement processing, orthe recording processing may continue after the finish of the movementprocessing.

After the finish of the movement processing, in a case where the usermoves the imaging apparatus 10 again to follow the subject J, and thedetector 60 detects the motion of the housing 14 at the time ofmovement, the movement processing is executed again, and the series ofprocessing (S006 to S017) is repeated.

(Flow of Processing in Case where Panning Mode is Selected)

The flow of processing in a case where the panning mode is selected willbe described with reference to FIGS. 15 and 16.

In a case where the panning mode is selected, a procedure (S021 to S024)from the start of the imaging apparatus 10 until execution of therecording processing by pushing the release button 310 by the user isthe same as in a case where the tracking mode is selected. That is, evenin a case where the panning mode is selected, the setting step and theextraction step are executed, and the recording step is started bypushing the release button 310.

In a case where the panning mode is selected, the user moves the imagingapparatus 10 to the first side of the horizontal direction. The detector60 detects the motion of the housing 14 at the time of movement, thatis, movement to the first side of the horizontal direction (S025). In acase where the detector 60 detects movement of the housing 14, theprocessing portion 30 shifts the extraction range to the second side ofthe horizontal direction by an amount corresponding to the movementamount of the housing 14 using the EIS (S026).

Then, the processing portion 30 executes the determination processingand determines whether or not the end of the extraction range on thesecond side reaches the end of the angle of view on the second side(S027). In a case where a determination that the end of the extractionrange on the second side reaches the end of the angle of view on thesecond side is made in the determination processing, the processingportion 30 executes the movement processing and moves the extractionrange toward the first side in the horizontal direction as illustratedin FIG. 7 (S028). Step S028 corresponds to the movement step.

In the first embodiment, the processing portion 30 sets the movementspeed of the extraction range in executing the movement processing andmoves the extraction range at the set movement speed in the movementprocessing. The movement speed of the extraction range is set inaccordance with a horizontal movement time period tx. The horizontalmovement time period tx is a time period from when horizontal movementof the housing 14 detected by the detector 60 starts to when the end ofthe extraction range on the second side reaches the end of the angle ofview on the second side. For example, the horizontal movement timeperiod tx can be calculated from an angular velocity in horizontalmovement of the housing 14 detected by the detector 60, or a movementspeed in a case where the end of the extraction range on the second sidemoves toward the end of the angle of view on the second side. The speedof the extraction range during execution of the movement processing maybe constant or may be changed. For example, the user may change thespeed of the extraction range to any speed by operating the operatingpart 50.

In the movement processing, the processing portion 30 moves theextraction range and displays the extraction video during movement ofthe extraction range on the display 40 (refer to FIG. 8). At this point,the processing portion 30 may obtain a time period (that is, therequired time period for movement) from when the movement starts to whenthe end of the extraction range reaches the end of the angle of view,from the movement speed of the extraction range. Furthermore, theprocessing portion 30 may display the remaining movement time period trbased on the difference between the obtained required time period andthe elapsed time period from the start of movement on the display 40. Inaddition, in the recording processing executed at the same time as themovement processing, the processing portion 30 records the extractionvideo during movement of the extraction range in the movement processingon the recording medium. That is, the recording step of recording theextraction video during movement of the extraction range in the movementstep is executed.

In addition, in the panning mode, in a case where the detector 60detects the motion (specifically, a vibration or the like) of thehousing 14 during execution of the movement processing (S029), theprocessing portion 30 executes the electronic camera shake correctionprocessing or the optical camera shake correction processing (S030). Instep S030, the camera shake correction is performed on the extractionvideo during movement of the extraction range. Accordingly, theextraction video that changes along with movement of the extractionrange can be stabilized, and the extraction video can be smoothlychanged even in a case where a camera shake or the like occurs.

In addition, in the movement processing, the processing portion 30determines whether or not the moving extraction range approaches theangle-of-view end nearby area within the angle of view (S031). In a casewhere the extraction range approaches the angle-of-view end nearby area,the processing portion 30 executes the alert processing (S032).Accordingly, the user can be notified that the extraction rangeapproaches near the end of the angle of view.

In a case where the end of the extraction range on the first sidereaches the end of the angle of view on the first side by horizontalmovement of the extraction range, or in a case where any of the finishconditions (J1) and (J2) is established (S033 and S034), the processingportion 30 finishes the movement processing. At this point, therecording processing may be finished in accordance with the finish ofthe movement processing, or the recording processing may continue afterthe finish of the movement processing.

After the finish of the movement processing, in a case where the userhorizontally moves the imaging apparatus 10 again to the first side, andthe detector 60 detects the motion of the housing 14 at the time ofmovement, the determination processing in step S027 is executed again.In a case where a determination that the end of the extraction range onthe second side reaches the end of the angle of view on the second sideis made in the determination processing, the movement processing isexecuted again, and the series of processing (S028 to S034) is repeated.

While description is not provided in the case illustrated in FIGS. 15and 16, in a case where the panning mode is selected, the zooming(specifically, the electronic zooming and the optical zooming) may beperformed on the video of the subject within the extraction video duringexecution of the movement processing. In this case, the movement speedof the extraction range may be adjusted in accordance with the zooming,and the extraction range may be moved at the movement speed adjusted inaccordance with the zooming in the subsequent movement processing.

OTHER EMBODIMENTS

While a specific embodiment of the imaging apparatus according to theembodiment of the present invention is illustratively described so far,the first embodiment is merely an example, and other embodiments areconsidered. For example, as illustrated in FIG. 17, an imaging apparatusmain body 12X of the imaging apparatus and an external monitor 70 may beconnected in a wired manner or a wireless manner.

The external monitor 70 is configured with a display device such as aliquid crystal monitor, a recorder including a monitor, or aninformation processing terminal including a monitor, such as a personalcomputer, a smartphone, and a tablet terminal.

In the configuration illustrated in FIG. 17, the processing portion 30transmits video signals of the extraction video to the external monitor70, and the extraction video is displayed on the external monitor 70side. In this case, the extraction video may be displayed on theexternal monitor 70 instead of displaying the extraction video on thedisplay 40 (display) of the imaging apparatus. Alternatively, theextraction video may be displayed on both of the display 40 of theimaging apparatus and the external monitor 70.

In addition, in the first embodiment, a case where the housing 14 movesin the horizontal direction, and the detector 60 detects movement of thehousing 14 in the horizontal direction as the motion of the housing 14is illustratively described. However, the present invention is notlimited thereto, and the detector 60 may detect movement of the housing14 in an up-down direction. In this case, the processing portion 30 maymove the extraction range in the movement processing in accordance withmovement of the housing 14 in the up-down direction detected by thedetector 60.

In addition, in the first embodiment, while the extraction range slidesin the horizontal direction in the movement processing in a case wherethe panning mode is selected, the present invention is not limitedthereto. For example, the extraction range may slide in the up-downdirection in the movement processing.

In addition, in the first embodiment, in a case where the tracking modeis selected, and the user moves the imaging apparatus 10 in order totrack the subject, the detector 60 detects the motion of the housing 14at the time of movement. The processing portion 30 automaticallyexecutes the movement processing in connection with detection of themotion of the imaging apparatus 10 and moves the extraction range totrack the subject. However, the present invention is not limitedthereto. The user may set the subject within the extraction video as thetracking target by operating a touch panel or the like, and the movementprocessing may be executed based on an operation of issuing aninstruction to start processing by the user. Hereinafter, such a casewill be described in detail as a second embodiment of the presentinvention.

A basic configuration of an imaging apparatus according to the secondembodiment is approximately the same as the imaging apparatus 10 and isillustrated in FIG. 3. That is, the imaging apparatus according to thesecond embodiment comprises the imaging portion 20 and the processingportion 30 (processor). The processing portion 30 executes the settingprocessing, the extraction processing, the movement processing, thezooming processing, and the recording processing.

In the setting processing, the extraction range is set within the angleof view based on contents of initial setting or an instruction operationof the user. The extraction processing is the same processing as theabove extraction processing. In this processing, the extraction videocaptured within the extraction range is extracted. In the secondembodiment, for example, the user tracks the subject within theextraction video while changing the angle of view by moving the imagingapparatus, and designates the subject by touching the subject on thetouch panel. Accordingly, the designated subject is set as the trackingtarget.

In the movement processing, the extraction range is moved over time suchthat the subject of the tracking target enters within the extractionrange. By executing such movement processing, the video of the subjectis included at all times in the extraction video during movement of theextraction range. In addition, in the movement processing, theextraction range is moved, and the extraction video during movement ofthe extraction range is displayed on the display 40. In the secondembodiment, in a case where the user performs a predetermined buttonoperation or the like after designating the subject of the trackingtarget, the processing portion 30 executes the movement processing basedon the predetermined button operation as a trigger.

The zooming processing includes the electronic zooming processing andthe optical zooming processing. In any of the zooming processing, thezooming is performed on the video of the subject within the extractionvideo. For example, in a case where the subject video ratio changes dueto movement of the subject in the depth direction, the processingportion 30 executes the electronic zooming processing in order to changethe size of the extraction range in accordance with the distance betweenthe subject and the imaging apparatus. In addition, in a case where theimage quality is decreased due to coarse image quality of the extractionvideo by performing the electronic zooming processing, the processingportion 30 executes the optical zooming processing in order to improvethe decreased image quality or displays the notification prompting theuser to perform the optical zooming processing on the display 40.

In the recording processing, the extraction video during movement of theextraction range in the movement processing is recorded on the recordingmedium.

In a case where the zooming processing is executed during the movementprocessing, the size of the extraction range, that is, the number ofpixels of the extraction video, changes before and after the zooming.Meanwhile, it is assumed that the movement speed of the extractionrange, that is, the number of pixels through which the extraction rangepasses per unit time period, is the same before and after the zooming.In this case, the movement speed of the extraction range looks rapidlychanging for the user who checks the extraction video through thedisplay 40.

In the second embodiment, in a case where the zooming processing isexecuted during the movement processing, the processing portion 30 inthe movement processing after the zooming processing moves theextraction range at the movement speed corresponding to the zooming(specifically, the electronic zooming or the optical zooming) performedin the zooming processing. Accordingly, a situation in which themovement speed of the extraction range looks rapidly changing due to thezooming can be avoided.

The movement processing of the second embodiment is not limited to acase where the extraction range is moved in order to track the subject.For example, the movement processing may be applied to a case where theextraction range slides from one end to the other end of the angle ofview, that is, a case where the panning mode is selected. Even duringexecution of such movement processing, the zooming processing can beexecuted. In the movement processing after the zooming processing, theprocessing portion 30 may move the extraction range at the movementspeed corresponding to the zooming.

According to the second embodiment described so far, an imaging methodincluding a setting step, an extraction step, a zooming step, a movementstep, and a recording step can be implemented as an imaging method usingthe imaging apparatus including the imaging portion 20 that captures thereference video which is a motion picture. In the setting step, theextraction range smaller than the angle of view is set within the angleof view in a case of capturing the reference video using the imagesensor 150. In the extraction step, the extraction video of theextraction range is extracted from the reference video. In the zoomingstep, the zooming (the optical zooming or the electronic zooming) isperformed on the video of the subject within the extraction video. Inthe movement step, the extraction range within the angle of view ismoved over time at the movement speed corresponding to the zoomingperformed in the zooming step. In the recording step, the extractionvideo during movement of the extraction range in the movement step isrecorded on the recording medium.

EXPLANATION OF REFERENCES

-   -   10: imaging apparatus    -   12, 12X: imaging apparatus main body    -   14: housing    -   20: imaging portion    -   30: processing portion    -   40: display    -   50: operating part    -   60: detector    -   70: external monitor    -   110: lens unit    -   112: zoom lens    -   114: focus lens    -   120: lens drive portion    -   130: stop portion    -   140: shutter    -   150: image sensor    -   160: A/D converter    -   210: control processing portion    -   220: video processing portion    -   230: internal memory    -   240: memory card    -   250: buffer    -   260: card slot    -   310: release button    -   320: zoom lever    -   330, 340: operation dial    -   350: cross button    -   C: vertical axis

What is claimed is:
 1. An imaging apparatus comprising: an image sensorthat captures a reference video which is a motion picture; a housingthat accommodates the image sensor; a detector for detecting a motion ofthe housing; and a processor configured to execute setting processing ofsetting an extraction range smaller than an angle of view within theangle of view in a case of capturing the reference video, extractionprocessing of extracting an extraction video within the extraction rangefrom the reference video, movement processing of moving the extractionrange within the angle of view over time in accordance with the motiondetected by the detector, and recording processing of recording theextraction video during movement of the extraction range in the movementprocessing on a recording medium, wherein the processor is configuredto, in a case where the detector has detected movement of the housing toa first side of a horizontal direction as the motion, move theextraction range to the first side over time.
 2. The imaging apparatusaccording to claim 1, further comprising: a display that displays theextraction video, wherein the processor is configured to, in themovement processing, move the extraction range and display theextraction video during movement of the extraction range on the display.3. The imaging apparatus according to claim 1, wherein the processor isconfigured to, in a case where the detector has detected the motion,execute determination processing of determining whether or not an end ofthe extraction range reaches an end of the angle of view, and theprocessor is configured to execute the movement processing in a casewhere a determination that the end of the extraction range reaches theend of the angle of view is made in the determination processing.
 4. Theimaging apparatus according to claim 3, wherein the processor isconfigured to, in a case where the detector has detected movement of thehousing to the first side of a horizontal direction as the motion,determine whether or not an end of the extraction range positioned on asecond side opposite to the first side reaches an end of the angle ofview positioned on the second side in the determination processing, andthe processor is configured to move the extraction range to the firstside in a case where a determination that the end of the extractionrange reaches the end of the angle of view is made in the determinationprocessing.
 5. The imaging apparatus according to claim 4, wherein theprocessor is configured to, in the movement processing, move theextraction range at a movement speed corresponding to a time period fromwhen the movement starts to when the end of the extraction range reachesthe end of the angle of view.
 6. The imaging apparatus according toclaim 1, wherein the processor is configured to, in a case where thedetector has detected the motion, select a subject captured within theextraction video before and after detection of the motion as a trackingtarget and, in the movement processing, move the extraction range suchthat the subject enters within the extraction range.
 7. The imagingapparatus according to claim 6, wherein the processor is configured to,in a case where the subject deviates from the angle of view duringexecution of the movement processing, finish the movement processing. 8.The imaging apparatus according to claim 6, wherein the processor isconfigured to, in the movement processing, move the extraction rangewhile changing a size of the extraction range based on a ratio of a sizeof a video of the subject to the size of the extraction range.
 9. Theimaging apparatus according to claim 1, wherein the processor isconfigured to, in a case where the detector has detected the motion ofthe housing during execution of the movement processing, further executecamera shake correction processing of performing camera shake correctionon the extraction video during movement of the extraction range.
 10. Theimaging apparatus according to claim 1, wherein the processor isconfigured to, in a case where the detector has detected the motion ofthe housing in a direction opposite to a movement direction of theextraction range during execution of the movement processing, finish themovement processing.
 11. The imaging apparatus according to claim 1,wherein the processor is configured to further execute zoomingprocessing of performing zooming on a video of a subject within theextraction video, and in a case where the zooming processing is executedduring execution of the movement processing, adjust a movement speed ofthe extraction range in accordance with the zooming in the movementprocessing.
 12. The imaging apparatus according to claim 2, wherein theprocessor is configured to, in a case where the detector has detectedthe motion, execute determination processing of determining whether ornot an end of the extraction range reaches an end of the angle of view,and the processor is configured to execute the movement processing in acase where a determination that the end of the extraction range reachesthe end of the angle of view is made in the determination processing.13. The imaging apparatus according to claim 2, wherein the processor isconfigured to, in a case where the detector has detected the motion,select a subject captured within the extraction video before and afterdetection of the motion as a tracking target and, in the movementprocessing, move the extraction range such that the subject enterswithin the extraction range.
 14. An imaging apparatus comprising: animage sensor that captures a reference video which is a motion picture;and a processor configured to execute setting processing of setting anextraction range smaller than an angle of view within the angle of viewin a case of capturing the reference video, extraction processing ofextracting an extraction video within the extraction range from thereference video, zooming processing of performing zooming on a video ofa subject within the extraction video, movement processing of moving theextraction range within the angle of view over time at a movement speedcorresponding to the zooming performed in the zooming processing, andrecording processing of recording the extraction video during movementof the extraction range in the movement processing on a recordingmedium.
 15. The imaging apparatus according to claim 14, furthercomprising: a display that displays the extraction video, wherein theprocessor is configured to, in the movement processing, move theextraction range and display the extraction video during movement of theextraction range on the display.
 16. The imaging apparatus according toclaim 14, wherein the zooming processing is processing of performingelectronic zooming on the video of the subject by changing a size of theextraction range in accordance with a distance between the imagingapparatus and the subject within the extraction video, and the processoris configured to, in the movement processing, move the extraction rangeat a movement speed corresponding to the electronic zooming.
 17. Theimaging apparatus according to claim 14, wherein the zooming processingis processing of performing optical zooming on the video of the subjectwithin the extraction video by moving a lens of the imaging apparatusalong an optical axis, and the processor is configured to, in themovement processing, move the extraction range at a movement speedcorresponding to the optical zooming.
 18. The imaging apparatusaccording to claim 15, wherein the zooming processing is processing ofperforming electronic zooming on the video of the subject by changing asize of the extraction range in accordance with a distance between theimaging apparatus and the subject within the extraction video, and theprocessor is configured to, in the movement processing, move theextraction range at a movement speed corresponding to the electroniczooming.
 19. The imaging apparatus according to claim 15, wherein thezooming processing is processing of performing optical zooming on thevideo of the subject within the extraction video by moving a lens of theimaging apparatus along an optical axis, and the processor is configuredto, in the movement processing, move the extraction range at a movementspeed corresponding to the optical zooming.
 20. An imaging method usingan imaging apparatus including an image sensor that captures a referencevideo which is a motion picture, the imaging method comprising: asetting step of setting an extraction range smaller than an angle ofview within the angle of view in a case of capturing the referencevideo; an extraction step of extracting an extraction video within theextraction range from the reference video; a movement step of moving theextraction range within the angle of view over time in accordance with amotion of the imaging apparatus detected by a detector; and a recordingstep of recording the extraction video during movement of the extractionrange in the movement step on a recording medium.